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The MAILING DATE of this communication appears on the cover sheet with the corresp ndence address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
eamed patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )^ Responsive to communication{s) filed on 8-25-2000 . 
2a)n This action is FINAL. 2b)^ This action is non-final. 

3) 0 Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 
Disposition of Claims 

4) ^ Claim(s) 1-20 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration, 

5) n Claim(s) is/are allowed. 

6) S Claim(s) 1-20 is/are rejected. 
/)□ Claim(s) is/are objected to. 

8) n Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) S The specification is objected to by the Examiner. 

1 0)n The drawing(s) filed on is/are: a)n accepted or b)^ objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 
1 1 )n The proposed drawing correction filed on is: a)D approved b)D disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) 0 The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§119 and 120 

13) ^ Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 

a)IElAII b)n Some*c)n None of: 

1 Certified copies of the priority documents have been received. 

2.n Certified copies of the priority documents have been received in Application No. . 



3.n Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) 0 Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 19(e) (to a provisional application). 

a) D The translation of the foreign language provisional application has been received. 

15) 0 Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121. 
Attachment(s) 

1 ) K Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-41 3) Paper No(s). . 

2) im Notice of Draftsperson's Patent Drawing Review (PTO-948) 5) [U Notice of Infomnal Patent Application (PTO-152) 

3) O Infomiation Disclosure Statement(s) (PTO-1449) Paper No(s) . 6) dl Other: 
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DETAILED ACTION 

Specification 

1 . Applicant is reminded of the proper content of an abstract of the disclosure. 

A patent abstract is a concise statement of the technical disclosure of the patent 
and should include that which is new in the art to which the invention pertains. If the 
patent is of a basic nature, the entire technical disclosure may be new in the art, and the 
abstract should be directed to the entire disclosure. If the patent is in the nature of an 
improvement in an old apparatus, process, product, or composition, the abstract should 
include the technical disclosure of the improvement. In certain patents, particularly 
those for compounds and compositions, wherein the process for making and/or the use 
thereof are not obvious, the abstract should set forth a process for making and/or use 
thereof. If the new technical disclosure involves modifications or alternatives, the 
abstract should mention by way of example the preferred modification or alternative. 

The abstract should not refer to purported merits or speculative applications of 
the invention and should not compare the invention with the prior art. 

Where applicable, the abstract should include the following: 

(1) if a machine or apparatus, its organization and operation; 

(2) if an article, its method of making; 

(3) if a chemical compound, its identity and use; 

(4) if a mixture, its ingredients; 

(5) if a process, the steps. 

Extensive mechanical and design details of apparatus should not be given. 

2. The abstract of the disclosure is objected to because the abstract fails to provide 
summary 50 to 100 word summary of the invention. Further, the abstract fails to 
provide an explanation of the invention that is understandable by one with ordinary skill 
in the art. The abstract appears to be a literal translation that is not understandable to 
one with ordinary skill in the art. The Examiner cannot determine the intent of the 
Applicant for phrases and words within the abstract. These phrases and words include 
at least the following: "...easily understand relations between characters and changes 
in relations ..." and "relations". For the purpose of Examination, the Examiner interprets 
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these words as follows: "relations" as meaning relationships. Correction is required. 
See MPEP§ 608.01(b). 

3. A substitute specification excluding the claims is required pursuant to 37 CFR 
1 .125(a) because the current specification is not understandable to one with ordinary 
sl<ill in the art. The current specification appears to be a literal translation that is not 
understandable to one with ordinary skill in the art. Some but not necessarily all 
examples of such language that is not understandable includes the following: "video 
data structure for video browsing based on relations," cited in page 1, lines 4-6, 
cannot be comprehended using ordinary skill in the art; therefore, for the purpose of 
examination, the Examiner interprets the above as the data structure for storing video 
files; "...represented or classified into format chunk, index chunk, media chunk, 
segment chunk, target chunk, and/or representation chunk," cited in page 1, lines 16 - 
19, cannot be comprehended by using ordinary skill in the art because the word, 
"chunk" is subject to various interpretations. Therefore, the Examiner for the purpose of 
Examination interprets the word "chunk" as a large noteworthy amount. "Accordingly, a 
user can select an object through a table and reproduce for display a particular segment 
where the object is shown in the video," as cited in page 1 , lines 22 - 25, cannot be 
understood by one with ordinary skill in the art because the Examiner is unsure how the 
Applicant intends to apply the words, "object" and "segment" to the rest of the sentence; 
therefore, the Examiner interprets the word "object" as being either a window or an icon 
while interpreting the word "segment" as being a video clip. "The constant relation and 
variable relation between characters may also be browsed and displayed as a constant 
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relation and a variable relation between every characters related with a selected 
character," as cited by the Applicant on page 5, lines 19-23 cannot be understood by 
one with ordinary skill in the art because the Examiner is unable to determine the intent 
of the Applicant for the word "relation". For the purpose of examination, the word 
"relation" is interpreted as meaning a relationship. Also, the citation, "between every 
characters related with a selected character" cannot be comprehended using ordinary 
skill in the art. "Furthermore, the video browser of the present invention is characterized 
in that event segments showing constant relation and variable relation between 
characters is displayed as a key frame" as cited by the Applicant in page 6, lines 4-8 
cannot be comprehended using ordinary skill in the art. Therefore, for the purpose of 
examination, the Examiner interprets the sentence as meaning a video browser for 
presenting an invention that is characterized in the event segment while showing a 
constant relation and a variable relation between the characters that are displayed as a 
key frame. "The event/object relation graph DS 1 19 describes a constant relation or 
changes in relation between characters, a relation between object and place, or a 
relation between object and event" as cited by the Applicant on page 8, lines 20 - 24 
contains the word "relation". The Examiner is unable to determine the users intent for 
the word relation. For the purpose of Examination, "relation" means "relationship". The 
"... relation graph DS 119 includes is organized into an entity relation 120 with a return 
which allows a display of character relations in a tree structure ..." as cited by the 
Applicant on page 9, lines 22 - 25 cannot be understood by one with ordinary skill in the 
art because the Examiner is unable to interpret the intent by the Application for writing 
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the word "entity relation." "Namely, key frames of main event segments which directly 
show a selected relation or key frames of secondary event segments which indirectly 
show the selected relation" as cited by the Applicant on page 13, 19-22 cannot be 
understood by one using ordinary skill in the art because the Examiner is unable to 
determine the intent of the Applicant for using the words, "main event segments" and 
"selected relation." The Examiner interprets "main event segments" as video clips 
associated with the primary output event and the "selected relation" as a user selected 
relationship. The citation by the application, "displayed by connecting the time DS 1 13 
of the segment DS 1 1 2 with the time DS 1 1 1 of the level 1 1 0 in the key frame view DS 
106 of the visual DS 102" on page 14, lines 22 - 24 cannot be comprehended using 
ordinary skill in the art because the Examiner is unable to interpret the intent for the 
Applicant of the phrase, "key frame view DS 106 of the visual DS 102." "Furthermore, 
the data structure for browsing a video is based on a relation between characters and a 
variable relation between them" as cited by the Applicant on page 16, lines 18-20 
cannot be understood using ordinary skill in the art because the Examiner is unable to 
determine the intent of the Applicant for the phrase, "and a variable relation between 
them." For the purpose of examination, the Examiner interprets the above citation as a 
data structure for browsing video data that is based on the relationships between 
various characters. 

A substitute specification filed under 37 CFR 1 .125(a) must only contain subject 
matter from the original specification and any previously entered amendment under 37 
CFR 1.121. If the substitute specification contains additional subject matter not of 
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record, the substitute specification must be filed under 37 CFR 1 .125(b) and must be 
accompanied by: 1 ) a statement that the substitute specification contains no new 
matter; and 2) a marked-up copy showing the amendments to be made via the 
substitute specification relative to the specification at the time the substitute 
specification is filed. 

4. A substitute specification in proper idiomatic English and in compliance with 37 
CFR 1 .52(a) and (b) is required. A statement that it contains no new matter must 
accompany the substitute specification filed. 



Claim Rejections - 35 USC §112 

5. The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 

The specification shall conclude with one or more clainns particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

6. Claims 1 - 20 are rejected under 35 U.S.C. 112, second paragraph, as being 
indefinite for failing to particularly point out and distinctly claim the subject matter which 
applicant regards as the invention. 

7. Claims 12-14 recites the limitation "DS" in "structure DS" as cited in claim 12. 
There is insufficient antecedent basis for this limitation in the claim. The Examiner 
interprets "DS" as meaning a data structure for the purpose of Examination. 

8. The claims are generally narrative and indefinite, failing to conform with current 
U.S. practice. They appear to be a literal translation into English from a foreign 
document and are replete with grammatical and idiomatic errors. Some but not 
necessarily all examples of such errors include the following: 
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As per claims 1 and 15, reading, "... a main screen which displays video 
segments corresponding to significant events showing a relation between two displayed 
characters of said character screen according to a user selection, wherein said relation 
may be constant or variable" as cited in claims 1 and 15, lines 4-8 cannot be 
understood by one with ordinary skill in the art. The Examiner cannot determine the 
intent by the Applicant for the following word: "relation". For the purpose of 
Examination, the Examiner interprets the word "relation" as being a relationship. 

As per claim 2, "...said video segments by a summary data of said video 
segments" as cited in claim 2, line 2 cannot be understood by one with ordinary skill in 
the art. The Examiner cannot determine the intent by the Applicant for the following 
phrase: "summary data". For the purpose of Examination, the Examiner interprets the 
phrase "summary data" as being a summary list. 

As per claim 12, "...wherein the highlight view DS is organized into multiple 
levels which enables a display of multi-levels of highlight data and wherein the key 
frame view DS is organized into multiple levels which enables a display of multi-levels of 
summarized data" as cited in claim 12, lines 8-11 cannot be understood by one with 
ordinary skill in the art. The Examiner cannot determine the intent by the Applicant for 
the following words and phrases: "multi-levels" and "key frame view DS". For the 
purpose of Examination, the Examiner interpret the word "multi-levels" as being multiple 
level and the phrase "key frame view DS" as being the view of the main window. 

As per claim 13, "...each actual video segment data within a video data" as cited 
in claim 12, lines 4-7 cannot be understood by one with ordinary skill in the art. The 
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Examiner cannot determine the intent by the Applicant for the following words and 
phrases: "each actual video segment data" and "within a video data". For the purpose 
of Examination, the Examiner interprets the phrase "each actual video segment data" as 
being each video segment and the phrase "within a video data" as being a video file. 

The Examiner recommends that the Applicant employ a professional translator 
who is familiar with ordinary spoken and written English when reviewing, editing, and 
amending the claims. 

Claim Rejections - 35 USC § 102 

9. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(a) the invention was known or used by others in this country, or patented or described in a printed 
publication In this or a foreign country, before the invention thereof by the applicant for a patent. 

10. Claims 1 - 5 and 12 - 14 are rejected under 35 U.S.C. 102(a) as being 
anticipated by Schein et al. (US 6,323,911). 

This rejection is made on the claims as best understood by the Examiner. 
Claim 1: 

Schein teaches a video browsing system (fig. 4a, #102). Schein teaches a 
character screen which displays characters of the video (fig. 4a, #126). Any 
representation of a human being or an alphanumeric symbol is considered a 
representation of a character. Schein teaches a main screen which displays video 
segments corresponding to significant events showing a relation between two displayed 
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characters of said character screen according to a user selection, wherein said 
selection may be constant or variable (fig. 4a, #126). 
Claim 2: 

Schein teaches a main screen displaying video segments by summary data of 
said video segments (fig. 4a, #126). 
Claim 3: 

Schein teaches a user selecting two characters through the character screen to 
display the video segments corresponding to significant events showing a relation 
between the two displayed characters (fig. 4a, #1 10). When the user selects a 
program, representations corresponding to the relationships between characters are 
also selected. 
Claim 4: 

Schein teaches a character relation-variable event screen which displays 
connections between variable relations and significant events (fig. 4a, #110). 
Claim 5: 

Schein teaches a character-variable event screen displaying either one or both 
the variable relations and events by key frames (fig. 4a, #110). 
Claim 12: 

Schein teaches a video browsing system (fig. 4a). Schein teaches a visualization 
DS which includes a highlight view DS for displaying event data by a highlight and a key 
frame view DS for displaying event data as key frames, wherein the highlight view DS is 
organized intp multiple levels which enable a display of multi-levels of highlight data and 



Application/Control Number: 09/645,61 3 Page 1 0 

Art Unit: 2174 

wherein the key frame view DS is organized into multiple levels which enables a display 
of multi-levels of summarized data (fig. 4a, #110). Schein teaches a syntactic structure 
DS which includes information for displaying actual video segments of a video (fig. 4a, 
#110). Schein teaches a semantic structure DS which includes additional information 
describing a video (fig. 4a). Readable information uses the semantic structure DS for 
providing additional information. 
Claim 13: 

Schein teaches a syntactic structure DS organized into a segment DS including 
actual video segment data and a time DS including corresponding temporal positions of 
each actual video segment data within a video data (fig. 4a, #126). 
Claim 14: 

Schein teaches a semantic structure DS organized into sub-level structures (fig. 
4a). Rows demonstrate various levels. 

Schein teaches an event DS which includes event information, wherein said 
event DS is organized into sub-level structures of a reference to segment which 
includes reference information necessary for displaying a video segment of a video 
corresponding to the event selected by a user. Further, an annotation DS includes 
information that connects said selected events which actual positions of said selected 
events within video data and information explaining said selected events (fig. 4a, #110). 

Schein teaches an object which includes object information (fig. 4a, #32). Schein 
teaches an event/relation graph DS which includes information on at least one of 
constant relations between objects, variable relations between objects, or relations 
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between objects and events, wherein said event/object relation graph DS is organized 
into an entity relation with sub-level structures of a Reference to Object which connects 
objects having either a constant relation or variable relation, a Reference to Event that 
connects events which are significant to a relation between each connected objects, 
and a relation which includes information on a nature and title of relation between each 
connected objects (fig. 4a, #102). 

Claim Rejections - 35 USC § 103 

1 1 . The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not Identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

12. Claims 6 -11 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Schein et al. (US 6,323,91 1 ) as applied to claim 1 above, and further in view of Yeo et 
al. (US 5,831,945). 

Claim 6: 

Schein teaches a relation screen, which displays constant relations, and variable 
relations between characters selected from the character screen and related characters 
(fig. 4a). 

5 

Schein fai^to teach a tree structure. Yeo teaches constant relations and variable " 
relations being displayed in a tree structure (fig. 4). Yeo teaches a main scene screen 
which displays significant events corresponding to one of either a constant relation or 
variable relation selected from the relation screen (fig. 4). It would have been obvious 
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to one with ordinary skill in the art at the time of the invention to combine the tree 
structure taught by Yeo with the EPG disclosed by Schein. Doing so enables the user 
to view various hierarchies within the EPG. 
Claim 7: 

Yeo teaches a constant relation being displayed on a top level of said tree-like 
structure and variable relations being displayed on lower levels of said tree-like 
structure (fig. 4). 
Claim 8: 

Schein teaches a main scene screen displaying the significant events by key 
frames (fig. 4a, #32). The icons represent key frames. 
Claim 9: 

Schein teaches a selection screen wherein the main scene screen displaying 
either one or both main significant events or secondary significant events corresponding 
to said relation selected from the relation screen, according to a user selection through 
said selection screen (fig. 4a, #102). 
Claim 10: 

Schein teaches a selection screen wherein the relation screen displays constant 
relations and variable relations corresponding a relation type selected by a user through 
the said selection screen (fig. 4a, #102). 
Claim 11: 
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Yeo teaches a relation type may be one of a family relation, a business relation, 
or a social relation (fig. 4). The use of icons represents various family, business, and 
social relations. 

13. Claims 15-20 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Schein et al. (US 6,323,91 1 ) and Yeo et al. (US 5,831 ,945). 
Claim 15: 

Schein teaches video browsing (fig. 4a, #102). Schein teaches a character 
screen, which displays characters of a video (fig. 4a, #126). Schein teaches a main 
screen which displays video segments corresponding to significant events showing a 
relation between two displayed characters of said character screen according to a user 
selection, wherein said relation may be constant or variable (fig. 4a, #126). 

Schein fails to teach a tree structure. Yeo teaches a relation screen which 
displays constant relations and variable relations between a character selected from the 
character screen and related characters, wherein said constant relations and variable 
relations are displayed in a tree-like structure (fig. 4). It would have been obvious to 
one with ordinary skill in the art at the time of the invention to combine the tree structure 
taught by Yeo with the EPG disclosed by Schein. Doing so enables the user to view 
various hierarchies within the EPG. 
Claim 16: 
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Yeo teaches a constant relation being displayed on a top level of the said tree- 
like structure and variable relations are displayed on lower levels of the said tree 
structure (fig. 4). 
Claim 17: 

Schein teaches a character screen, the main screen, the relation screen, and the 
main scene screen being displayed using a video data structure (fig. 4a). 

Schein teaches a visualization DS which includes a highlight view DS for display 
for displaying event data by a highlight and a key frame view DS for displaying event 
data as key frames, wherein the view DS for displaying event data as key frames, 
wherein the highlight view DS is organized into multiple levels which enables a display 
of multi-levels of highlight data and wherein the key frame view DS is organized into 
multiple levels which enables a display of multi-levels of summarized data (fig. 4a, 
#110). 

Schein teaches a syntactic structure DS which includes information for displaying 
actual video segments of a video (fig. 4a, #110). 

Schein teaches a semantic structure DS which includes additional information 
describing a video (fig. 4a). Readable information uses the semantic structure DS for 
providing additional information. 

Claim 18: 

Schein teaches a selection screen wherein the main scene screen displays either 
one or both main significant events or secondary significant events corresponding to 
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said relation selected from the relation screen, according to a user selection through 
said selection screen (fig. 4a, #102). 
Claim 19: 

Schein teaches a selection screen wherein the relation screen displaying 
constant relations and variable relations corresponding a relation type selected by a 
user through said selection screen (fig. 4a, #102). 
Claim 20: 

Yeo teaches that a relation type may be one of a family relation, a business 
relation, or a social relation (fig. 4). 

Conclusion 

14. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Thomas J Joseph whose telephone number is 703-305- 
3917, The examiner can normally be reached Mondays through Fridays from 7:30 am 
to 4:00 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Kristine Kincaid can be reached on 703-308-0640. The fax phone numbers 
for the organization where this application or proceeding is assigned are 703-746-7239 
for regular communications and 703-746-7238 for After Final communications. 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the receptionist whose telephone number is 703-305- 
3900. 
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